Automatic graph classification

In educational measurement, the objective is to measure some unknown latent variable or variables that represent a student’s underlying ability or abilities in a subject domain. The first step in this process is to collect observable data from the student that will provide a window into these latent variables. Typically, this takes the form of a student’s response to a test question (usually called a test “item”). Multiple-choice has proven to be the most popular item format for large-scale assessments because of the ease of automatically scoring student responses. However, multiple-choice items provide only a limited view into a student’s knowledge and understanding, and the proliferation of test prep companies shows that multiple-choice items can be easily gamed. Because of these weaknesses, an ongoing avenue of research in educational measurement focuses on richer environments for assessing student knowledge, such as questions that ask students to highlight a phrase in text, drag items into the proper place in the periodic table, and create concept maps (Zenisky & Sireci, 2002). But many of these methods are simply glorified multiple-choice questions—they do not provide a truly open environment for students to produce responses. When it comes to scoring open-ended responses, the Educational Testing Service has made progress with a trio of projects; the e-rater and c-rater projects score essays and natural-language responses (Attali & Burstein, 2006; Leacock & Chodorow, 2003), and the m-rater project scores equations and graphs (Livingston, 2009). It is this latter problem—the problem of scoring a student’s graphical response to a mathematics question—that we will examine in this paper. A weakness of the m-rater system is that it asks a student to generate a graph using a proprietary graphing tool, and then it tests the correctness of the graph by converting the graph to an equation or testing points (Livingston, 2009). An arguably